{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Multiple features\n",
    "\n",
    "* Notation:\n",
    "   * $n$ = number features\n",
    "   * $x^{(i)}$ = input features of $i^{th}$ training example\n",
    "   * $x_{j}^{(i)}$ = value of feature $j$ in $i^{th}$ training example\n",
    "   \n",
    "* Hypothesis:\n",
    "\n",
    "$$\n",
    "x = \\left[\\begin{matrix} x_0 \\\\ x_1 \\\\ x_2 \\\\ ... \\\\ x_n \\end{matrix}\\right],\\quad (x_{0}=1)\\\\\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\theta = \\left[\\begin{matrix} \\theta_0 \\\\ \\theta_1 \\\\ \\theta_2 \\\\ ... \\\\ \\theta_n \\end{matrix}\\right]\\\\\n",
    "$$\n",
    "\n",
    "* means:\n",
    "\n",
    "for all emample:\n",
    "$$\n",
    "x_0^{(i)}=1,\\\\\n",
    "h_{\\theta}(x)=\\sum_{i=0}^{n}\\theta_{i}\\cdot x_{i}=\\theta^{T}x,\\quad (x_{0}=1)\\\\\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Gradient descent for multiple variables\n",
    "\n",
    "* notation:\n",
    "\n",
    "Hypothesis: $h_{\\theta}(x)=\\sum_{i=0}^{n}\\theta_{i}\\cdot x_{i}$\n",
    "\n",
    "Parameters: $\\theta_0,\\theta_1,\\theta_2,...,\\theta_n$\n",
    "\n",
    "Cost function: $J(\\theta_0, \\theta_1,...,\\theta_n)=\\frac{1}{2m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)})^2$\n",
    "\n",
    "Gradient descent:\n",
    "$\n",
    "\\quad Repeat\\{ \\\\ \n",
    "\\quad\\quad\\theta_{j}:=\\theta_{j}-\\alpha \\frac{\\partial}{\\partial\\theta_{j}}J(\\theta_0, \\theta_1,...,\\theta_n)\\quad (j=1,2,...n) \\\\ \n",
    "\\quad\\} \\\\ \n",
    "$ \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Feacture Scaling\n",
    "\n",
    "* Goal:Make sure the Gradient descent faster\n",
    "\n",
    "![](../pic/QQ截图20201022093551.png)\n",
    "\n",
    "* Get every feature into range:$-1\\leq x_i \\leq 1$\n",
    "\n",
    "* Mean normalization（均值归一化）\n",
    "\n",
    "$$x_i := \\frac{x_i - \\bar{x}}{\\max (x)-\\min (x)}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Learning rate\n",
    "\n",
    "* How to make sure gradient descent is working correctly?\n",
    "\n",
    "> 画出No.iterations - $J(\\theta)$的函数\n",
    ">\n",
    "> $J(\\theta)$ should decsease after every iteration\n",
    "\n",
    "\n",
    "* How to choose learning rate $\\alpha$?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 Features and polynomial regression\n",
    "\n",
    "* 多元线性回归\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5 Normal equation（正规方程）\n",
    "\n",
    "* 求导set = 0（n维：拉格朗日乘子法）\n",
    "\n",
    "\n",
    "![](../pic/QQ截图20201022145209.png)\n",
    "\n",
    "![](../pic/QQ截图20201022145544.png)\n",
    "\n",
    "![](../pic/QQ截图20201022150124.png)\n",
    "\n",
    "* $\\theta = (X^TX)^{-1}X^Ty$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.6 Normal equation and non-invertibility(optional)（正规方程）\n",
    "\n",
    "* the reasion of $X^TX$ is non-invertible(伪逆)\n",
    "\n",
    "![](../pic/QQ截图20201022151722.png)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
